Undergraduate Biology Lecture Courses Predominantly Test Facts about Science Rather than Scientific Practices.

Scientific practices are the skills used to develop scientific knowledge and are essential for careers in science. Despite calls from education and government agencies to cultivate scientific practices, there remains little evidence of how often students are asked to apply them in undergraduate courses. We analyzed exams from biology courses at 100 institutions across the United States and found that only 7% of exam questions addressed a scientific practice and that 32% of biology exams did not test any scientific practices. The low occurrence of scientific practices on exams signals that undergraduate courses may not be integrating foundational scientific skills throughout their curriculum in the manner envisioned by recent national frameworks. Although there were few scientific practices overall, their close association with higher-order cognitive skills suggests that scientific practices represent a primary means to help students develop critical thinking skills and highlights the importance of incorporating a greater degree of scientific practices into undergraduate lecture courses and exams.

How Administration Stakes and Settings Affect Student Behavior and Performance on a Biology Concept Assessment.

Biology instructors use concept assessments in their courses to gauge student understanding of important disciplinary ideas. Instructors can choose to administer concept assessments based on participation (i.e., lower stakes) or the correctness of responses (i.e., higher stakes), and students can complete the assessment in an in-class or out-of-class setting. Different administration conditions may affect how students engage with and perform on concept assessments, thus influencing how instructors should interpret the resulting scores. Building on a validity framework, we collected data from 1578 undergraduate students over 5 years under five different administration conditions. We did not find significant differences in scores between lower-stakes in-class, higher-stakes in-class, and lower-stakes out-of-class conditions, indicating a degree of equivalence among these three options. We found that students were likely to spend more time and have higher scores in the higher-stakes out-of-class condition. However, we suggest that instructors cautiously interpret scores from this condition, as it may be associated with an increased use of external resources. Taken together, we highlight the lower-stakes out-of-class condition as a widely applicable option that produces outcomes similar to in-class conditions, while respecting the common desire to preserve classroom instructional time.

Access to Online Formative Assessments in Lower-Division Undergraduate Biology Courses: Investigating Barriers to Student Engagement.

Instructors use a variety of online formative assessment (FA) activities to support learning outside class. Previous studies have revealed barriers for students in online courses, but little is known about the barriers students experience when completing online FA assignments. Understanding these barriers to access is critical to fostering more inclusive learning for all students. Using a framework from previous work in online learning, we examined student perceptions of online FA access with respect to five barrier categories: technical resources, instructor organization, social interactions, personal engagement, and learning environment. We developed and administered a survey to more than 1200 undergraduate biology students at 2-year and 4-year institutions. Students responded to statements using Likert scales and open-ended prompts. Statistical models indicated differences in access across the barrier categories and revealed that demographic characteristics were associated with certain barrier categories. Furthermore, technical resources, instructor organization, and personal engagement barriers were associated with lower course performance. In open-ended responses, students most frequently suggested that changes to scheduling logistics, course delivery, and FA format would improve their online FA experience. We discuss how these findings and student suggestions can inform instruction, particularly how instructors can alter their FA characteristics to better suit their student populations.

Revisiting Clickers: In-Class Questions Followed by At-Home Reflections Are Associated with Higher Student Performance on Related Exam Questions.

Clicker questions are a commonly used active learning technique that stimulates student interactions to help advance understanding of key concepts. Clicker questions are often administered with an initial vote, peer discussion, and a second vote, followed by broader classroom explanation. While clickers can promote learning, some studies have questioned whether students maintain this performance on later exams, highlighting the need to further understand how student answer patterns relate to their understanding of the material and to identify ways for clickers to benefit a broader range of students. Systematic requizzing of concepts during at-home assignments represents a promising mechanism to improve student learning. Thus, we paired clicker questions with at-home follow-up reflections to help students articulate and synthesize their understandings. This pairing of clickers with homework allowed us to decipher how student answer patterns related to their underlying conceptions and to determine if revisiting concepts provided additional benefits. We found that students answering both clicker votes correctly performed better on isomorphic exam questions and that students who corrected their answers after the first vote did not show better homework or exam performance than students who maintained an incorrect answer across both votes. Furthermore, completing the follow-up homework assignment modestly boosted exam question performance. Our data suggest that longer-term benefits of clickers and associated homework may stem from students having repeated opportunities to retrieve, refine, and reinforce emerging conceptions.

Am I getting through? Surveying students on what messages they recall from the first day of STEM classes.

BACKGROUND: The first day of class helps students learn about what to expect from their instructors and courses. Messaging used by instructors, which varies in content and approach on the first day, shapes classroom social dynamics and can affect subsequent learning in a course. Prior work established the non-content Instructor Talk Framework to describe the language that instructors use to create learning environments, but little is known about the extent to which students detect those messages. In this study, we paired first day classroom observation data with results from student surveys to measure how readily students in introductory STEM courses detect non-content Instructor Talk. RESULTS: To learn more about the instructor and student first day experiences, we studied 11 introductory STEM courses at two different institutions. The classroom observation data were used to characterize course structure and use of non-content Instructor Talk. The data revealed that all instructors spent time discussing their instructional practices, building instructor/student relationships, and sharing strategies for success with their students. After class, we surveyed students about the messages their instructors shared during the first day of class and determined that the majority of students from within each course detected messaging that occurred at a higher frequency. For lower frequency messaging, we identified nuances in what students detected that may help instructors as they plan their first day of class. CONCLUSIONS: For instructors who dedicate the first day of class to establishing positive learning environments, these findings provide support that students are detecting the messages. Additionally, this study highlights the importance of instructors prioritizing the messages they deem most important and giving them adequate attention to more effectively reach students. Setting a positive classroom environment on the first day may lead to long-term impacts on student motivation and course retention. These outcomes are relevant for all students, but in particular for students in introductory STEM courses which are often critical prerequisites for being in a major. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40594-021-00306-y.

GenBio-MAPS as a Case Study to Understand and Address the Effects of Test-Taking Motivation in Low-Stakes Program Assessments.

The General Biology-Measuring Achievement and Progression in Science (GenBio-MAPS) assessment measures student understanding of the Vision and Change core concepts at the beginning, middle, and end of undergraduate biology degree programs. Assessment coordinators typically administer this instrument as a low-stakes assignment for which students receive participation credit. While these conditions can elicit high participation rates, it remains unclear how to best measure and account for potential variation in the amount of effort students give to the assessment. To better understand student test-taking motivation, we analyzed GenBio-MAPS data from more than 8000 students at 20 institutions. While the majority of students give acceptable effort, some students exhibited behaviors associated with low motivation, such as low self-reported effort, short test completion time, and high levels of rapid-selection behavior on test questions. Standard least-squares regression models revealed that students' self-reported effort predicts their observable time-based behaviors and that these motivation indices predict students' GenBio-MAPS scores. Furthermore, we observed that test-taking behaviors and performance change as students progress through the assessment. We provide recommendations for identifying and filtering out data from students with low test-taking motivation so that the filtered data set better represents student understanding.

Teaching Metabolism in Upper-Division Undergraduate Biochemistry Courses using Online Computational Systems and Dynamical Models Improves Student Performance.

Understanding metabolic function requires knowledge of the dynamics, interdependence, and regulation of metabolic networks. However, multiple professional societies have recognized that most undergraduate biochemistry students acquire only a surface-level understanding of metabolism. We hypothesized that guiding students through interactive computer simulations of metabolic systems would increase their ability to recognize how individual interactions between components affect the behavior of a system under different conditions. The computer simulations were designed with an interactive activity (i.e., module) that used the predict-observe-explain model of instruction to guide students through a process in which they iteratively predict outcomes, test their predictions, modify the interactions of the system, and then retest the outcomes. We found that biochemistry students using modules performed better on metabolism questions compared with students who did not use the modules. The average learning gain was 8% with modules and 0% without modules, a small to medium effect size. We also confirmed that the modules did not create or reinforce a gender bias. Our modules provide instructors with a dynamic, systems-driven approach to help students learn about metabolic regulation and equip students with important cognitive skills, such as interpreting and analyzing simulation results, and technical skills, such as building and simulating computer-based models.

Making a First Impression: Exploring What Instructors Do and Say on the First Day of Introductory STEM Courses.

Student impressions formed during the first day of class can impact course satisfaction and performance. Despite its potential importance, little is known about how instructors format the first day of class. Here, we report on observations of the first day of class in 23 introductory science, technology, engineering, and math (STEM) courses. We first described how introductory STEM instructors structure their class time by characterizing topics covered on the first day through inductive coding of class videos. We found that all instructors discussed policies and basic information. However, a cluster analysis revealed two groups of instructors who differed primarily in their level of STEM content coverage. We then coded the videos with the noncontent Instructor Talk framework, which organizes the statements instructors make unrelated to disciplinary content into several categories and subcategories. Instructors generally focused on building the instructor-student relationship and establishing classroom culture. Qualitative analysis indicated that instructors varied in the specificity of their noncontent statements and may have sent mixed messages by making negatively phrased statements with seemingly positive intentions. These results uncovered variation in instructor actions on the first day of class and can help instructors more effectively plan this day by providing messages that set students up for success.

Innovative teaching knowledge stays with users.

Programs seeking to transform undergraduate science, technology, engineering, and mathematics courses often strive for participating faculty to share their knowledge of innovative teaching practices with other faculty in their home departments. Here, we provide interview, survey, and social network analyses revealing that faculty who use innovative teaching practices preferentially talk to each other, suggesting that greater steps are needed for information about innovative practices to reach faculty more broadly.

Benefits of a College STEM Faculty Development Initiative: Instructors Report Increased and Sustained Implementation of Research-Based Instructional Strategies.

The Summer Institutes on Scientific Teaching (SI) is a faculty development workshop in which science, technology, engineering, and mathematics (STEM) instructors, particularly from biology, are trained in the Scientific Teaching (ST) pedagogy. While participants have generally reported positive experiences, we aimed to assess how the SI affected participants' teaching practices. Building on a previously developed taxonomy of ST practices, we surveyed SI participants from the 2004-2014 SI classes regarding specific ST practices. Participants' self-reported use and implementation of ST practices increased immediately after SI attendance as well as over a longer time frame, suggesting that implementation persisted and even increased with time. However, instructors reported implementation gains for some practices more than others. The practices with the highest gains were engaging students in their own learning, using learning goals in course design, employing formative assessment, developing overarching course learning goals, representing science as a process, and facilitating group discussion activities. We propose that the ST practices showing the greatest gains may serve as beneficial focal points for professional development programs, while practices with smaller gains may require modified dissemination approaches or support structures.

Interactive learning modules with 3D printed models improve student understanding of protein structure-function relationships.

Ensuring undergraduate students become proficient in relating protein structure to biological function has important implications. With current two-dimensional (2D) methods of teaching, students frequently develop misconceptions, including that proteins contain a lot of empty space, that bond angles for different amino acids can rotate equally, and that product inhibition is equivalent to allostery. To help students translate 2D images to 3D molecules and assign biochemical meaning to physical structures, we designed three 3D learning modules consisting of interactive activities with 3D printed models for amino acids, proteins, and allosteric regulation with coordinating pre- and post-assessments. Module implementation resulted in normalized learning gains on module-based assessments of 30% compared to 17% in a no-module course and normalized learning gains on a comprehensive assessment of 19% compared to 3% in a no-module course. This suggests that interacting with these modules helps students develop an improved ability to visualize and retain molecular structure and function.

Resources for Teaching and Assessing the Vision and Change Biology Core Concepts.

The Vision and Change report called for the biology community to mobilize around teaching the core concepts of biology. This essay describes a collection of resources developed by several different groups that can be used to respond to the report's call to transform undergraduate education at both the individual course and departmental levels. First, we present two frameworks that help articulate the Vision and Change core concepts, the BioCore Guide and the Conceptual Elements (CE) Framework, which can be used in mapping the core concepts onto existing curricula and designing new curricula that teach the biology core concepts. Second, we describe how the BioCore Guide and the CE Framework can be used alongside the Partnership for Undergraduate Life Sciences Education curricular rubric as a way for departments to self-assess their teaching of the core concepts. Finally, we highlight three sets of instruments that can be used to directly assess student learning of the core concepts: the Biology Card Sorting Task, the Biology Core Concept Instruments, and the Biology-Measuring Achievement and Progression in Science instruments. Approaches to using these resources independently and synergistically are discussed.

"What Will I Experience in My College STEM Courses?" An Investigation of Student Predictions about Instructional Practices in Introductory Courses.

The instructional practices used in introductory college courses often differ dramatically from those used in high school courses, and dissatisfaction with these practices is cited by students as a prominent reason for leaving science, technology, engineering, and mathematics (STEM) majors. To better characterize the transition to college course work, we investigated the extent to which incoming expectations of course activities differ based on student demographic characteristics, as well as how these expectations align with what students will experience. We surveyed more than 1500 undergraduate students in large introductory STEM courses at three research-intensive institutions during the first week of classes about their expectations regarding how class time would be spent in their courses. We found that first-generation and first-semester students predict less lecture than their peers and that class size had the largest effect on student predictions. We also collected classroom observation data from the courses and found that students generally underpredicted the amount of lecture observed in class. This misalignment between student predictions and experiences, especially for first-generation and first-semester college students and students enrolled in large- and medium-size classes, has implications for instructors and universities as they design curricula for introductory STEM courses with explicit retention goals.

Tools for Change: Measuring Student Conceptual Understanding Across Undergraduate Biology Programs Using Bio-MAPS Assessments.

Assessing learning across a biology major can help departments monitor achievement of broader program-level goals and identify opportunities for curricular improvement. However, biology departments have lacked suitable tools to measure learning at the program scale. To address this need, we developed four freely available assessments-called Biology-Measuring Achievement and Progression in Science or Bio-MAPS-for general biology, molecular biology, ecology/evolution, and physiology programs. When administered at multiple time points in a curriculum, these instruments can provide departments with information on how student conceptual understanding changes across a major and help guide curricular modifications to enhance learning.

Student Understanding of DNA Structure-Function Relationships Improves from Using 3D Learning Modules with Dynamic 3D Printed Models.

Understanding the relationship between molecular structure and function represents an important goal of undergraduate life sciences. Although evidence suggests that handling physical models supports gains in student understanding of structure-function relationships, such models have not been widely implemented in biochemistry classrooms. Three-dimensional (3D) printing represents an emerging cost-effective means of producing molecular models to help students investigate structure-function concepts. We developed three interactive learning modules with dynamic 3D printed models to help biochemistry students visualize biomolecular structures and address particular misconceptions. These modules targeted specific learning objectives related to DNA and RNA structure, transcription factor-DNA interactions, and DNA supercoiling dynamics. We also designed accompanying assessments to gauge student learning. Students responded favorably to the modules and showed normalized learning gains of 49% with respect to their ability to understand and relate molecular structures to biochemical functions. By incorporating accurate 3D printed structures, these modules represent a novel advance in instructional design for biomolecular visualization. We provide instructors with the materials necessary to incorporate each module in the classroom, including instructions for acquiring and distributing the models, activities, and assessments. © 2019 International Union of Biochemistry and Molecular Biology, 47(3):303-317, 2019.

GenBio-MAPS: A Programmatic Assessment to Measure Student Understanding of Vision and Change Core Concepts across General Biology Programs.

The Vision and Change report provides a nationally agreed upon framework of core concepts that undergraduate biology students should master by graduation. While identifying these concepts was an important first step, departments also need ways to measure the extent to which students understand these concepts. Here, we present the General Biology-Measuring Achievement and Progression in Science (GenBio-MAPS) assessment as a tool to measure student understanding of the core concepts at key time points in a biology degree program. Data from more than 5000 students at 20 institutions reveal that this instrument distinguishes students at different stages of the curriculum, with an upward trend of increased performance at later time points. Despite this trend, we identify several concepts that advanced students find challenging. Linear mixed-effects models reveal that gender, race/ethnicity, English-language status, and first-generation status predict overall performance and that different institutions show distinct performance profiles across time points. GenBio-MAPS represents the first programmatic assessment for general biology programs that spans the breadth of biology and aligns with the Vision and Change core concepts. This instrument provides a needed tool to help departments monitor student learning and guide curricular transformation centered on the teaching of core concepts.

Phys-MAPS: a programmatic physiology assessment for introductory and advanced undergraduates.

We describe the development of a new, freely available, online, programmatic-level assessment tool, Measuring Achievement and Progress in Science in Physiology, or Phys-MAPS ( http://cperl.lassp.cornell.edu/bio-maps ). Aligned with the conceptual frameworks of Core Principles of Physiology, and Vision and Change Core Concepts, Phys-MAPS can be used to evaluate student learning of core physiology concepts at multiple time points in an undergraduate physiology program, providing a valuable longitudinal tool to gain insight into student thinking and aid in the data-driven reform of physiology curricula. Phys-MAPS questions have a modified multiple true/false design and were developed using an iterative process, including student interviews and physiology expert review to verify scientific accuracy, appropriateness for physiology majors, and clarity. The final version of Phys-MAPS was tested with 2,600 students across 13 universities, has evidence of reliability, and has no significant statement biases. Over 90% of the physiology experts surveyed agreed that each Phys-MAPS statement was scientifically accurate and relevant to a physiology major. When testing each statement for bias, differential item functioning analysis demonstrated only a small effect size (<0.008) of any tested demographic variable. Regarding student performance, Phys-MAPS can also distinguish between lower and upper division students, both across different institutions (average overall scores increase with each level of class standing; two-way ANOVA, P < 0.001) and within each of three sample institutions (each ANOVA, P ≤ 0.001). Furthermore, at the level of individual concepts, only evolution and homeostasis do not demonstrate the typical increase across class standing, suggesting these concepts likely present consistent conceptual challenges for physiology students.

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates.

A new assessment tool, Ecology and Evolution-Measuring Achievement and Progression in Science or EcoEvo-MAPS, measures student thinking in ecology and evolution during an undergraduate course of study. EcoEvo-MAPS targets foundational concepts in ecology and evolution and uses a novel approach that asks students to evaluate a series of predictions, conclusions, or interpretations as likely or unlikely to be true given a specific scenario. We collected evidence of validity and reliability for EcoEvo-MAPS through an iterative process of faculty review, student interviews, and analyses of assessment data from more than 3000 students at 34 associate's-, bachelor's-, master's-, and doctoral-granting institutions. The 63 likely/unlikely statements range in difficulty and target student understanding of key concepts aligned with the Vision and Change report. This assessment provides departments with a tool to measure student thinking at different time points in the curriculum and provides data that can be used to inform curricular and instructional modifications.

Student, instructor, and observer agreement regarding frequencies of scientific teaching practices using the Measurement Instrument for Scientific Teaching-Observable (MISTO).

BACKGROUND: The Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports. Understanding the growth and impact of ST requires instruments to accurately measure the extent to which practitioners implement ST in their courses. Researchers have typically relied on students, instructors, or observers to document course teaching practices, but it remains unclear whether and how these perspectives differ from each other. To address this issue, we modified our previously published instrument to generate the Measurement Instrument for Scientific Teaching-Observable (MISTO), which can be completed by students, instructors, and observers, and we investigated the degree of similarity between these three perspectives across 70 undergraduate science courses at seven different institutions in the USA. RESULTS: We found that the full MISTO and Active Learning subcategory scores showed the highest correlations among the three perspectives, but the degree of correlation between perspectives varied for the other subcategories. Match scores between students and instructors were significantly higher than observer matches for the full MISTO and for the Active Learning, Inclusivity, and Responsiveness subcategories. CONCLUSIONS: We find that the level and type of agreement between perspectives varies across MISTO subcategories and that this variation likely stems from intrinsic differences in the course access and scoring decisions of the three perspectives. Building on this data, we recommend MISTO users consider their research goals, available resources, and potential artifacts that may arise when deciding which perspective best fits their needs in measuring classroom teaching practices.